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Conducting a Spanish National Adult Literacy Survey: 
A Discussion of the Issues 



L Introduction/Rationale for Seminar 

The following is a "discussion paper" based on a seminar hosted by the Educational Testing 
Service (ETS) Southern California Field Office (SCFO), on the feasibility of conducting a Spanish version 
of the National Adult Literacy Survey (NALS). 

The SCFO decided to host this seminar because of the importance and timeliness of this topic. 
Data indicate that the non-English speaking population in the United States is growing and represents 
sizable numbers. Of the 31.8 million Americans who speak a language other than English, 17 million 
(54%) speak Spanish. This represents an increase of over 6 million since 1980 (Census, 1990; Macfas, 
1993). Spanish is the prevailing non-English language spoken in 39 states. Thus, the data show that a 
significant number of Americans (14%) speak a non-English language and our country may not be 
realizing the potential of a major segment of our society if we do not examine their literacy levels and 
resources. 

Another important reason for the seminar relates to the capacity and ability of literacy service 
providers to accommodate and educate the growing numbers of non-English speaking adults. A recent 
publication of the U.S. Department of Education Office of Vocational and Adult Education (1991) notes 
the changing enrollments in federally funded programs. The report points out: 

"Of the three components of adult education programs - Adult Basic 
Education, English as a Second Language, and Adult Secondary 
Education - ESL experienced the largest increase in enrollment, from 19 
percent of total enrollment in 1980 to 34 percent in 1989 ... " (p. 14-15) 

Although the increase in non-English speaking adults has resulted in increases in ABE (Adult 
Basic Education) and ESL (English as a Second Language) enrollments in adult schools (as well as 
community and library-based programs), very little is known about their native language literacy levels 
and/or potential. Methodologies used in the past resulted in limited data on literacy levels. For example, 
studies usually examined self-reports on bilingual proficiency (ability to speak a second language), and 
not on reading and writing abilities -- two significant elements of literacy. 

A third rationale for the seminar centers on the need to determine in more precise ways the 
literacy levels of non-English speakers in either English or their native language. For example, 
determining the extent of literacy of Spanish speaking adults has been difficult. The recent NALS report 
(Kirsch, et al., 1993) makes note of the fact that "twenty-five percent of the respondents who performed 
in this level (level I) were immigrants who may have been just learning to speak English." However, 
we do not know to what extent language played a role in their low English literacy levels, nor do we have 
any direct measures of their native language literacy levels. 

We do know that adults who speak primarily another language other than English, do not do well 
on English language tests. For example, Vargas (1988) reports on data from the English Language 
Proficiency Survey (ELPS) that found that "... while 13% of the population 20 years of age and older 
was illiterate, 48% of adults whose native language was other than English were illiterate." 

1 
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The National Chicano Survey (NCS) conducted in 1979 (and described by Macfas, 1988) is 
unique in that it includes data on Spanish language and literacy (written use in the household) as well as 
on English. Data were gathered from a national sample of Mexican Origin households (with about 1000 
respondents 17 years old and over) which utilized a bilingual questionnaire and interviewers as data 
collection methodologies. These data indicate that "... 26% of the Mexican origin population was not 
literate in English or Spanish in 1979. Approximately 32% reported literacy abilities only in English, 
while 20% reported biliteracy in English and Spanish, and 22% reported literacy only in Spanish" 
(Macfas, 1988). 

Using the number of school years completed for those 18 years and older as a surrogate literacy 
measure, different results emerge. "Of those who are classified as English illiterate ... 35% were literate 
in Spanish only" (Macfas, 1988). Thus, the potential number of Spanish speaking adults who are literate 
in their native language can be relatively high, even though they have little English language proficiency. 

Much of the data on non-English language literacy is self-reported, and self-reported data can 
yield mixed results. For example, Vargas (1988) reports that self-reported data used on the English 
Language Proficiency Survey (ELPS) found that 86% of adults from non-English backgrounds who were 
illiterate in English reported being illiterate in their native language as well. Stated differently, 14% were 
literate in Spanish compared to Macfas* reported 22%. One explanation for the difference might be the 
fact that Macfas analyzed the NCS which consisted of Mexican Origin respondents, whereas the ELPS 
data examined Hispanic - with no subgroup breakdown. 

One major improvement in this area comes from the current ETS NALS survey (Kirsch et al., 
1993, mentioned above) which provides valuable information on self-reported literacy levels in a non- 
English language. In this survey, adults (oversampled for Hispanics) reported how well they felt they 
could speak, read and write in their native language. 1 Although still using self-reported data, this 
methodological design is a substantial first step since it includes speaking, reading and writing as literacy 
outcomes. Now the discussion has turned to the feasibility of conducting a direct assessment of adults 
who speak a non-English language -- in this case Spanish. 2 

The reader is reminded that this discussion paper raises the many issues associated with 
conducting a national adult literacy survey in the Spanish language. Since no divect national assessment 
of this kind had ever been conducted, participants tended to cover several issues and propose many ideas 
on how, why, and where such an assessment would take place. In fact, several significant ideas are 
presented in the following pages that will begin to set the framework and discussion for a Spanish version 
of the NALS. However, in the short time span that participants were together, (a day and a half), in- 
depth analyses of any one concern or area was difficult - and several questions need further examination. 



1 A special report titled "Literacy and Cultural Diversity" is forthcoming as part of the larger NALS report series. 

2 It should be noted that a "NALS type" assessment has already been conducted in the French language in Canada 
(a representative from this study was in attendance). 
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n. Organization and Purpose of the Seminar 



In the Spring of 1992, the ETS Southern California Field Office (SCFO) hosted a two-day 
seminar to discuss issues related to developing a Spanish version of the National Adult Literacy Survey 
(NALS). The SCFO staff wanted to address linguistic, political, and logistic concerns related to such an 
effort at this seminar. Adult literacy program evaluators and researchers, demographers, practitioners 
and policy makers were invited to share their thoughts on this topic. Individuals with experience and 
expertise in national adult literacy surveys were in attendance as well. Participants came from Mexico 
and Canada, as well as the United States, to discuss issues related to second language assessment of adults 
- in this case - the Spanish National Adult Literacy Survey. 3 

With this group of participants, ETS was able to begin to generate a framework for a Spanish 
NALS, and based on information gathered for this framework, provide recommendations that would assist 
in the development of a Spanish version of the NALS. 

Since the discussion for a Spanish language assessment centered around methodologies used by 
ETS in past and present national surveys, it was important to describe ETS* efforts in this area to seminar 
participants. ETS SCFO staff felt that the current NALS survey would add tremendously to our 
understanding of a national assessment effort of this magnitude. As such, the Project Officer for the 
current National Adult Literacy Survey conducted by Educational Testing Service (Princeton) was in 
attendance and provided this information. 

In an attempt to learn from previous efforts outside the U.S., ETS staff invited a representative 
from Canada where a "NALS type" assessment was conducted in October of 1989 by Statistics Canada 
on behalf of the National Literacy Secretariat. Also, feeling that a Spanish adult literacy assessment 
would be of great interest to Mexico, a representative from the Instituto Nacidnal de Educacidn para 
Adultos (INEA) was invited. These two guests added an international point of view to the discussions. 

Although the experience of the current NALS certainly guided our discussion, there were specific 
concerns about developing a Spanish version of the NALS that ETS staff wanted to explore in more 
detail. They are presented briefly below. 

Purposes and Uses of Spanish NALS . What would be the purpose of a national survey 
using Spanish as the language of assessment? 

Issues in Task (Item) Development . What would the tasks/items look like? Would they 
be translations of the current tasks/items? What "Spanish" would be used in the items? 
What would be the content, structure, and nature of the tasks? 

Spanish NALS Relationship to Other ETS Assessments . How would a Spanish version 
of the NALS "equate" or "relate" to other national assessments using similar constructs? 
How would Spanish items on the three literacy scales i.e., prose, document, quantitative, 
relate to items from the English NALS version. 



3 A full listing of seminar participants is provided in the Appendix section of this paper. 
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Issues Related to Pretesting & Sampling . 
Spanish speakers only? Total population? 



Who would be sampled? Latinos only? 



Issues Related to Scaling . Would Spanish language tasks distribute on the three (prose, 
document, quantitative) NALS scales similarly to English tasks; and if so, how would 
they relate to each other? 

Although several of these issues overlapped, we tried to identify elements specifically related to 
each category. Further, participants spent more time on purposes (bilingual, Spanish assessment); nature 
of tasks (translations of current tasks or contextual to Spanish speaking population); and sampling (all 
Spanish speakers no matter what race/ethnicity, all Latinos, all Spanish speaking Latinos); than on the 
other categories (e.g., Item Response Theory and scaling). 

These categories were organized into a Participant Feedback Form where seminar participants 
wrote comments on each section. Thus, even though participants didn't cover each category in equal 
depth during the discussion, they were still able to write their comments on the Participant Feedback 
Form (See Appendix). 

III. Results of Discussion on a Spanish National Adult Literacy Survey 

As described above, five themes were identified as discussion starters for the seminar: 

1 . Purposes and Uses of Spanish NALS 

2. Issues in Task (Item) Development 

3. Spanish NALS Relationship to Other ETS Assessments 

4. Issues Related to Pretesting & Sampling 

5. Issues Related to Scaling 

Each section is reviewed below. 

1 . Purposes and Uses of Spanish NALS 

This topic received a lot of attention at the seminar. All were in agreement that a Spanish 
"version" of NALS would be a significant and valuable undertaking. Practitioners and researchers alike 
saw the value of such an assessment and its positive impact not only on the nation's ABE and ESL 
programs, but for society in general. The discussion that followed represented various points of view. 
This issue was discussed from the perspective of practitioners, policy makers, researchers and program 
evaluators. 

Practitioners 

Practitioners were concerned with how such an assessment would help in establishing better 
instructional programs and developing appropriate curriculum. In this light, participants mentioned a 
variety of purposes that a Spanish NALS could serve. Even though we asked participants to "brainstorm" 
on possible uses of a Spanish version of the NALS, it obviously could not serve all of the purposes 
proposed in this section. Nonetheless all suggestions are presented. 
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Participants pointed out that a Spanish NALS assessment could be used to make curriculum 
placement decisions or assist in designing instruction. In the latter case, much of the grammar-based ESL 
materials could be replaced or supplemented by more demanding cognitive skill-based materials if 
assessments showed high levels of native language literacy. This means that a Spanish NALS could 
provide information about the placement of adults in native language curriculums/materials. 

Such an assessment could also assist practitioners with ESL placement concerns. For example, 
one participant wondered if such an assessment could answer the question, "When is an ESL student no 
longer an ESL student?" In this case, such an assessment could inform providers on how to transfer from 
native language instruction to second language (i.e., English) instruction. 

Determining the effects of first language on second language ?cquisition was considered an 
important by-product of a Spanish NALS. For instance, the Spanish NALS could assist practitioners "in 
determining the role of primary language literacy development on second language acquisition." 
Thresholds could be established in the native language that indicate the learner is ready for transitioning 
to English language tasks. 

The topic of code-switching came up and its implications for designing instruction. In this case, 
determining language dominance was considered important, however, the simultaneous use of two 
languages was viewed as an intriguing concept warranting further examination. The potential of a 
"biliterate assessment" using the NALS constructs were explored considering the impact of code- 
switching. 

Participants mentioned using the Spanish NALS as a Spanish language assessment in crder to 
determine literacy levels in Spanish. This concept is similar to the current purpose of the NALS, which 
is to determine the English literacy levels of America's adult population. By using the current NALS 
framework, Spanish language literacy could be determined using tasks (possibly with similar text 
processing demands) from the prose, document and quantitative scales to determine native language 
literacy levels. 

In sum, practitioners (i.e., literacy service providers) saw the potential of a Spanish NALS to 
inform curriculum decisions and in some cases drive the curriculum by focusing on relevant real-life tasks 
and materials. The assessment was seen as providing in-depth information for native language literacy 
instruction and ESL instruction. In fact, the assessment could inform research by providing data on how 
native language literacy affects second language development. Finally, participants saw the potential of 
the Spanish NALS in determining the literacy levels of Spanish speaking adults. 

Policy Makers 

This section discusses the importance of a Spanish NALS for policy-makers. Generally, 
participants felt that a Spanish NALS can impact several areas e.g., civil service, health, social services, 
education, etc. Participants agreed that a Spanish version of the NALS would be beneficial, however, 
mos* noted several key issues. The key issues centered around the idea that a storyline was important 
to present the idea to the American public. 

For example, one could argue that in times of national (or state) emergencies, instructions and/or 
safety procedures would need to be made available to the public. It would be necessary for government 
officials as well as social workers, doctors, and civil service staff to have an idea of the language 
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capacities of the communities affected so they could communicate emergency procedures effectively to 
them. 

Secondly, participants felt that literacy should be viewed as a national resource. In this case, 
literacy in any language would be regarded as meaningful and a potential resource for a productive (e.g., 
working) community. By assessing native language literacy, one could get a better understanding not 
only of the nature of our language resources, but also begin to examine the "untapped" language 
resources in America as well. 

Along this same line is the idea of access for non-English speaking adults to community/national 
resources and their full participation in society. For example, one crucial area relating to participation 
in society is voting. The Voting Rights Act of 1965 - recently extended by the House and Senate - 
provides for voting ballots in a non-English language if "5% of the voting age citizens share a single 
foreign language, their literacy rate is lower that the national average and the Census Bureau finds that 
they have limited proficiency in English." 

Further, biliteracy can be seen as valuable in bridging the gap between monolinguals, promoting 
multicultural understanding among Americans, as well as being a valuable personal asset. Therefore, 
knowing the extent of the American population that speak more than one language would be one way to 
gauge our nation's language/literacy resources. 

Educational policy makers would be interested in determining how Spanish literacy levels impact 
second language literacy and/or biliteracy. This was discussed briefly above in the Practitioner section. 
Literacy providers would have a wealth of knowledge regarding curriculum development and instructional 
methodologies if they could assess not only native language literacy (i.e., Spanish), but better understand 
the underlying cognitive skills associated with each level of literacy, and how these skills transfer to the 
second language. 

Participants noted that estimating language literacy would have an impact on jobs as well. 
Generally, more and more of the workforce is becoming non-English speaking - especially in the 
southwest United States, Ascertaining the extent of workers* literacy levels is a major step in our 
understanding of the workforce literacy and job training needs. One seminar participant specializing in 
developing workplace training materials for non-English speaking workers found the idea of a Spanish 
assessment for adults important and relevant to workplace literacy programs. In this context, with the 
demographic characteristics of the workforce changing, we would need to broaden our understanding of 
literacy to include literacy in other languages, and determine the extent to which this information can 
inform instruction, training, materials development, and assessment. 

Knowing the extent of language resources could also affect the economic well-being of businesses 
as well. More aind more we see the multi-national aspect of marketing goods and services, especially in 
the wake of the North American Free Trade Agreement (NAFTA) and the potential growing Spanish 
language market. As both consumers and workers, non-English speaking adults can make a significant 
contribution to the U.S. economy. 

In sum, policy makers felt they could benefit from a Spanish language adult assessment in order 
to communicate with non-English speaking persons relative to health-related issues, or in assisting victims 
of natural catastrophes (e.g., earthquakes). Further, such an assessment could assist policy makers in 
making informed decisions about funding adult literacy programs for the limited English speaking (e.g., 
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language emphasis relating to ESL teaching methodologies), and informing workplace literacy 
instructional approaches. 4 Finally, with the NAFTA agreement a reality, bilingualism could have a 
positive economic impact on future business ventures in the United States. 

Researchers and Program Evaluators 

The research concerns included sampling, task design, language of assessment, and the issue of 
biliteracy. These areas are discussed in detail in the next section. 

2. Issues in Task (Item) Development 

Research on several adult literacy assessments conducted by ETS (Young Adult Literacy Survey, 
Job Training Partnership Act - Employment Service/Unemployment Insurance, National Adult Literacy 
Survey) suggest that task difficulty is determined by at least three features: content, structure, and 
purpose (Kirsch, 1993). Since the Spanish tasks would probably share some general constructs of the 
current NALS, it would be important to consider these aspects relative to the Spanish version. They are 
reviewed below. 

Content 

A point that came up often during the seminar was the idea that test developers need to consider 
the "language" and cultural aspects of tasks when constructing new ones in other languages. This aspect 
relates to the type of materials selected for the assessment tasks. The important point expressed was that 
materials be common to the experiences of the Spanish speaking group taking the assessments. The 
extent to which they are unfamiliar with the materials could increase the difficulty levels of the items. 

Structure 

Similar to content, the familiarity of the structure or format of the materials could cause tasks to 
be difficult. Currently the NALS provides various formats relative to prose tasks (dense text, text with 
headers and other organizers); document tasks (applications, different types of lists and tables, schedules, 
forms); and quantitative tasks (coupons, bill summaries, pay check stubs). These various formats were 
selected to replicate the types of text processing demands that adults in the United States come into 
contact. 

For Spanish speaking adults, Spanish language materials might have to be formatted differently 
to coincide with their unique characteristics. This may or may not be a problem - however, locating 
materials that have the varying formats represented in every day life would certainly be a goal for this 
assessment. 

The context of the tasks could also affect the format. The current NALS contains tasks from 
several life-related areas (e.g., work, home, community), and each setting contains materials that present 
different formats. With this in mind, one participant suggested that the Spanish NALS include "a number 
of items from all three categories (i.e., prose, documents, quantitative) within one context and equivalent 
items in other contexts, [then] select one context (e.g., recreation) and move across contexts to determine 



4 ETS has recently completed a report that begins to address this issue titled: Beyond the School Doors, 1992 
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ability to transfer." The point here is that the context cu the Spanish literacy tasks could determine the 
make-up or format of the prose samples, documents or quantitative demands; thus, these contexts must 
also be familiar to the Spanish speaking respondents. The ability to transfer across contexts would be 
an interesting research question to pursue. 

Purpose 

The demands for processing text vary from context to context. Sometimes adults have to merely 
match information from the question to the text which is identical or synonymous. In other cases, adults 
have to make inferences based on the text to successfully complete the task. The demands made on the 
adult to process text are related to task difficulty. With this in :nind, seminar participants were asked to 
elaborate on the nature of the tasks in the case of the Spanish NALS. 

Generally, participants felt that the demands of the tasks be similar to the demands that the 
Spanish speaking population are confronted with in their daily lives. One participant noted that the 
assessment should, "use Spanish in contexts which are natural to Spanish (not translated in from English 
items that would not occur in Spanish). There may be certain task demands that occur in only English 
language situations (or only in the United States) that do not have the same importance or purpose in 
Spanish. 

Language 

The language of the assessment was pretty much obvious in one sense - Spanish. However, 
there are other Spanish-speaking countries represented in the United States. Although their numbers do 
not compare to Mexico's, they nonetheless speak Spanish with variations in vocabulary and usage. 
Consequently, determining the "type" of Spanish to use for task development could be a complex matter. 
However, it is clear from recent U.S. Census data that the largest Spanish-speaking country represented 
in the U.S. is Mexico, and depending on the sampling design, "Spanish" representative of this group 
would probably be the most appropriate. In either case, the challenge would be to make the Spanish 
language selected for the assessment comprehensible to most if not all Spanish speaking respondents. 

The discussion also covered the perspective of "allowing for responses in either language (Spanish 
or English)." Further, suggestions were made that "code switching" (simultaneously changing from one 
language to another) should be permitted to display comprehension in whichever language [respondents] 
are most comfortable." The group discus;ed the logistical problems of such an assessment such as 
scoring and interpreting results. 

Having the tasks printed in two languages -- perhaps side-by-side - was raised as a possibility." 
In this case, respondents could decide in which language they preferred to respond. The notion of a 
locator test was suggested to determine language dominance - and the language of subsequent assessment 
tasks. 

In sum, seminar participants felt that Issues in Task Development were very impoitant ones to 
address and specifics and logistics still needed to be ironed-out. Generally, participants recommended 
that Spanish language tasks contain content that would be familiar to a Spanish speaking population and 
not mere translations of English tasks; that the format also be diverse and familiar to Spanish speaking 
adults vis a vis the content; that the purpose and demands of the tasks be consistent with demands that 
Spanish speaking adults face in their daily lives; and finally, that the Spanish language selected for the 
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assessment be suitable (comprehensible) to all Spanish speakers. In this latter case, proper translations 
for each Latino sub-population (e.g., Salvadoreno, Puerto Riqueno) was considered important, yet the 
logistics for implementing this idea still needed further examination. 

3. Spanish NALS Relationship to Other ETS Assessments 

ETS has conducted three major adult literacy surveys within the past decade: Young Adult 
Literacy Survey (YALS), 1986; Beyond The School Doors (JTPA/UI/ES), 1992; National Adult Literacy 
Survey (NALS), 1993. These ETS adult Jiteracy assessments share several characteristics. They all have 
three literacy scales, they use open-ended tasks, they contain real-life materials and text-processing 
demands that one encounters in daily situations, and they share the same definition of literacy — which 
ultimately guides the nature of the assessment. The definition is as follows: 

"Using printed and written information to function in society, to achieve 
one's goals, and to develop one's knowledge and potential. " 

Additionally, each of the ETS adult literacy assessments have been linked by using a subset of 
YALS tasks. In this case, comparisons of cohorts relative to various demographic characteristics have 
been made. 

Since a Spanish language version of the NALS could share several characteristics of these 
previous adult literacy assessments, including the definition of literacy, there may be an opportunity to 
link tasks from the scales, or explore the relationship between the different language tasks relative to their 
position on the three literacy scales. 

Spanish NALS/Current NALS 

The current NALS structure relative to the three literacy scales i.e., prose, document, and 
quantitative, would be beneficial for a Spanish version. This is not to limit the discussion of whether to 
add a written scale, and/or an oral language proficiency scale. In the former case, work has been 
initiated in scoring adult writing samples in the Spanish language (Sol6rzano, 1990). In the latter case, 
the Young Adult Literacy Survey (Kirsch & Jungeblut, 1986) contained oral language tasks organized 
to represent "different communication demands." Gathering oral language proficiency skill data from 
adults is made easier with surveys that include face-to-face interviews such as the NALS. In either case, 
the areas mentioned above could assist others in planning non-English writing and oral language 
assessments. 

If the Spanish NALS were to share many of the constructs of the current NALS, it would be 
beneficial to the discussion of biliteracy to analyze the Spanish tasks along the various difficulty levels 
from the three literacy scales to determine their relationship to similar English NALS tasks. 

This sentiment was echoed by one seminar participant who suggested the need to "devise 
metrices/methods to assure comparability and feasibility of studying/assessing biliteracy, or a literacy that 
may be context dependent rather than language dependent only." 
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Department of Labor (DOL) Study 
JTPA ES/UI Report 

This study consisted of a national sample of adults who were participants in the JTPA (Job 
Training Partnership Act), ES (Employment Services), and UI (Unemployment Insurance) programs. 
Data from this survey included profiles of adults' literacy skills from different occupations as well as 
educational, and demographic areas. 

Tasks containing text processing demands similar to those in this survey could be developed and 
results compared across scales, occupations, and language groups of adults. Various text processing 
demands associated with particular jobs can be identified and subsequent job training materials developed 
to coincide with these demands. 

Young Adult Literacy Survey (YALS) 

In 1986 ETS conducted the first large-scale assessment of Young Adults using the three literacy 
scale constructs (Kirsch & Jungeblut, 1986). As mentioned earlier, that assessment included the same 
guiding definition of adult literacy that the current NALS uses. In this case, the Spanish NALS might 
compare a 21-25 year old Spanish speaking cohort to those in the 1986 YALS study. Further, the ord 
language assessment used in this survey could also be explored as a basis for generating an oral language 
bilingual assessment. 

In sum, the potential linkages that such a Spanish NALS assessment can make to previous national 
studies using similar constructs (Young Adult Literacy Survey, 1986; Beyond the School Doors, 1992; 
National Adult Literacy Survey, 1992), could be beneficial. For this reason, there would be reason to 
explore the interrelatedness between Spanish and English versions of the National Adult Literacy Survey 
(NALS). Ultimately it would be beneficial to explore the similarity of text processing demands in two 
languages and identify the underlying cognitive (and metacognitive) skills associated with tasks at various 
levels on the three scales. This type of an analysis could have a profound impact on curriculum 
development in adult education programs and workplace-based training programs. 

4. Issues Related to Pretesting & Sampling of Spanish NALS 

Identifying Field Test Group 

Identifying the field test group - and eventually the national sample group - is basic to the goals 
of the Spanish NALS assessment. Seminar participants asked: "Who do we sample? All Latinos? Only 
LEP Latinos? All adults including Anglos who might speak Spanish?" Is this a national sample of 
Latinos' Spanish language capacities? Or, all American's capacities to speak Spanish? Some seminar 
participants felt tiiat this assessment should be an iissessment of America's language resources, while 
others felt it would be beneficial to assess "biliteracy" capacities of the Spanish speaking population. 

These are very important issues. As one seminar participant put it: "Who is the 'Spanish' 
speaker?" Another participant suggested that we "... design [the] study and sample either to assess 
Spanish literacy or assess Spanish and English literacy i.e., test entire sample in Spanish or test entire 
sample (or sub-sample) in both Spanish and English (except for individuals who report no proficiency in 
one). " It appeared that several participants diri not want to sample a "program specific" group (i.e., ESL 
adult learners), but rather to include a broader sample that would represent all of those who speak Spanish 
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(e.g., to include non-Hispanic). 



After much discussion, one theme that emerged was that the sample should be inclusive and 
representative of the American population. In this case, a sample stratified by certain language groups 
could be designed but allowing for all Americans to be represented. 

Data Bases for Sampling 

Probably the most comprehensive and most used data base in the country is the U.S. Census. 
The current NALS sampling sub-contractor (WESTSTAT) uses the Census data base for sampling and 
weighting. The Spanish NALS would also use the Census data base for stratifying the sample by 
population and language group. In the latter case, it would be important to determine the extent to which 
non-Hispanics speak Spanish and sample/stratify accordingly. The Census contains information on 
language spoken by race/ethnicity as well as self-reported data on literacy levels in the non-English 
language. One caveat to using the Census data is the underrepresentation of Latinos - especially the 
undocumented. 

Administration of Spanish NALS Items 

Since the Spanish NALS would be a direct assessment of Spanish language literacy levels, a 
household survey conducted by interview would be recommended. In this case however, interviewers 
conversant in both English and Spanish would be needed. The tasks would probably share many of the 
characteristics of the current NALS survey - that is - that they be open-ended and use simulated real-life 
tasks from several life-related contexts (e.g., home, community, work) that measure a broad range of 
information processing skills. 

Use of Bilingual Interviewers 

As mentioned above, the use of interviewers conversant in both English and Spanish would be 
necessary. Bilingual interviewers could be used to "screen" respondents who are not able to take the 
assessment in Spanish; or in the case of a bilingual assessment, that they administer the tasks in both 
languages. 

The sample design of the Spanish NALS is the centerpiece of the assessment. Determining who 
will be assessed frames the Spanish literacy issue. If one wants to address the issue of literacy among 
Spanish speakers, then sampling the Spanish speaking population only would be appropriate. If one wants 
to address the extent of Spanish use and literacy in America, then a stratified sampling procedure 
including all Americans could be used. Finally, if one wants to address the issue of biliteracy, then tasks 
need to be presented bilingually in a format (possibly English and Spanish tasks side-by-side) that allows 
respondents to answer in both languages. The use of bilingual interviewers during the administration of 
the tasks would be an important consideration in either case. 

5. Issues Related to Scaling 

The seminar group did not address the issues of scaling in any direct manner. In a general sense, 
participants recommended that "comparisons" be made between the Spanish NALS tasks and the current 
English NALS tasks. 
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Although participants did not address these issues directly, for purposes of this discussion paper, 
a brief discussion of the issues will be presented. 

IRT Methods and Equating 

The Item Response Theory (IRT) methods employed in the NALS would be suitable to a Spanish 
version and probably necessary to conduct any equating analyses to the current NALS. Some work has 
been done in this area. For example, a study was conducted by ETS that attempted to equate the Prueba 
de Aptitud Acad&nica (PAA) with the Scholastic Aptitude Test (SAT) (Angoff & Cook, 1988). These 
tests are given to similar respondents (secondary school students), however, one test is administered in 
English (SAT), and the other (PAA) in Spanish. This study can shed some light on the problems that 
can arise in bilingual equating. 

A number of issues about equating and translations across languages were raised during this 
investigation. For example, regarding "conversion of scores" between the PAA and the SAT the authors 
state: 

"Although the study of cultural differences has been of central interest to 
educators and social psychologists for many years, attempts to develop 
a deeper understanding of such differences have been frustrated by the 
absence of a common metric by which many such comparisons could be 
made. The reasons for this are clear. If two cultural groups differ from 
each other in certain ways that cast doubt on the validity of direct 
comparisons between them in other respects - if, for example, they differ 
sharply in language, custom, and values — then those very differences 
also will defy the construction of an unbiased metric by which we could 
hope to make such comparisons. * (p. 134-135) 

This suggests that care must be taken to insure that tasks be comparable in that the impact of 
language, custom, and values plays a minor discriminating role across languages. 

And regarding translations of the PAA and the SAT and making comparisons between culturally 
different groups the authors note: 

"... to provide a measuring instrument to make these comparisons, it is 
clearly insufficient simply to translate the test constructed for one 
language group into the language of the other, even with the adjustments 
in the items to conform to the more obvious cultural requirements of the 
second group. Nuances of expression and subtleties of custom are 
unwittingly embodied in words and phrases and often have particular 
meanings for one group but not for the other. It can therefore hardly be 
expected, without careful and detailed checks, that the translated items 
will have the same meaning and relative difficulty for the second group 
as they had for the original group before translation " (p. 135-136) 

The importance of language in assessment is evidenced in this statement. Language is not 
something that can be easily manipulated for research and/or comparison purposes. Care must be taken 
to conscientiously develop language-relevant items (tasks) that minimize language-based interference. 
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IV, Future Directions and Recommendations 

Several national and world-wide changes are taking place as the United States enters the twenty- 
first century. For example, the face of America is changing. As the past director of the Census Bureau 
notes, (quoted in Kirsch et. al., 1993), *lfyou gave America a face in 1990, it would have shown the first 
signs of wrinkles land] it would have been full of color. " Kirsch goes on to say: 

"Our increasing diversity can not only be seen but also heard: today, 
some 32 million individuals in the United States speak a language other 
than English, and these languages range from Spanish and Chinese to 
Yupik and Mon-Khmer. " 

The issue of language can be very political. For example, it can be seen as divisive where people 
use language differences to separate and prevent individuals from accessing resources and effectively 
contributing to society. On the other hand, the issue of language can be seen as a resource, where 
contrary to popular belief, language is used to show how people are similar in many respects; that they 
have the same aspirations for their children, that they want and need many of the same things as English 
speaking adults and are willing to contribute to the well-being of the United States. All adults bring 
resources to bear on the well-being of the nation, and language has all of the potential of being a unifying 
and valuable resource. 

Several issues still need to be examined relative to an effort of this type, however, based on the 
proceedings of the seminar, the following general recommendations are warranted. Seminar participants 
were not asked to specifically make recommendations, thus the author takes full responsibility for their 
content. 



Include a sampling procedure thai is stratified to include ail persons in the U.S. but weighted to 
insure a representative amount of Spanish speakers. 



In other words, that the sample not be program or race/ethnic specific, 
but that all persons in the U.S. who might speak Spanish be sampled - 
with assurances of sampling the major source of Spanish speaking adults. 



Develop tasks that reflect the Spanish speaking experience, and are not translations of the 
current NALS tasks. 



That the content, format and purpose of the tasks be consistent with what 
Spanish speaking persons in the U.S. might be confronted with in their 
daily lives. 
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Develop metrics that allow for future comparisons of the Spanish NALS with previous national 
adult literacy surveys. 



That Spanish tasks on the three literacy scales be comparable to English 
tasks that contain similar text processing demands contained in previous 
studies. Or, that methods be developed to determine the similarities of 
the two assessments. 

The first recommendation addresses language use as a national resource rather than divisive based 
on race or a particular program (e.g., ESL). The second recommendation recognizes the uniqueness of 
the Spanish language relative to content and structure, and warns against simplistic comparison/equating 
methodologies. The third recommendation attempts to contrast the text processing demands - and thus 
difficulty levels — between English and Spanish in order to determine potential transference of skills, 
bilinguaJity and literacy instruction. 

No doubt there are other issues and recommendations that can and will be proposed as discussions 
and perhaps the reality of a Spanish NALS becomes evident. The spirit of this paper is to share the ideas 
from a respected group of practitioners, researchers, and policy makers relative to an effort of this 
magnitude, and to raise important issues. This paper ultimately suggests that continued dialogue 
regarding the issues raised is necessary to embark on this important assessment venture. 
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